The present application relates to an image processing apparatus, an image processing method, and an image processing program for generating an image from an output signal of an image pickup element.
For an image pickup apparatus such as a digital camera, an image pickup element such as a CCD (charge coupled device) and a CMOS (complementary metal oxide semiconductor) is used. In such image pickup elements, a plurality of pixels are arranged in a planar form, and the pixels perform photoelectric conversion of light incident thereon (hereinafter, referred to as incident light) and output, as an electrical signal, the light that has been subjected to the photoelectric conversion. An arithmetic processing circuit performs arithmetic processing of the electrical signals, thereby generating an image.
Here, since the image pickup element as described above outputs an electrical signal in accordance with the intensity of light regardless of the wavelength of the incident light, it may be impossible to generate a color image by itself. In view of this, in an image pickup apparatus for generating a color image, color filters of colors which cause light having specific wavelength ranges (for example, RGB) to pass therethrough are provided to pixels, respectively (single panel system), and colors having wavelengths outside of the aforementioned wavelength ranges are attenuated by the color filters. With this structure, components of the incident light that have transmission wavelengths of the color filters reach the pixels and are converted into electrical signals.
There are various arrangements of the color filters of the respective colors. Generally, an arrangement called “Bayer arrangement” is employed. The Bayer arrangement refers to an arrangement that includes a column in which R and G are alternately disposed and a column in which B and G are alternately disposed so as not to set G to be adjacent to each other. Further, in the arrangement, the number of G, which has a high spectral sensitivity of a human eye, is set to twice the number of R and the number of B.
From the output signals of the pixels based on light that has passed through the color filters arranged as described above, the arithmetic processing circuit determines a pixel value (luminance value of each of RGB) of each of the pixels. At this time, because the pixels are provided with the color filters of the respective colors as described above, each of the luminance values calculated from the outputs corresponds to each transmission wavelength of the color filters, that is, relates to one of RGB. The arithmetic processing circuit calculates, from the outputs of adjacent image pickup elements, the luminance values of the other two colors by performing an “interpolation arithmetic”, thereby determining pixel values of the respective pixels. The interpolation arithmetic method affects an image quality, so various method are being studied.
For example, Japanese Patent Application Laid-open No. 2002-232904 (paragraph 0038, FIG. 5) (hereinafter, referred to as Patent Document 1) discloses a technique relating to the interpolation arithmetic. An “signal processing apparatus” disclosed in Patent Document 1 judges whether vertical stripes or horizontal stripes are included in an image or not on the basis of a correlation of a horizontal direction and a vertical direction with respect to a pixel to be interpolated, and in accordance with the judgment result, selects pixel data used for the interpolation. Thus, even in the case where the vertical stripes or the horizontal stripes exist in the image, the blur of the image can be corrected appropriately